Reduce memory usage of match all bitset #92777
Conversation
dnhatn
changed the title
dnhatn
added
v8.7.0
>enhancement
:Search/Search
|
Pinging @elastic/es-search (Team:Search) |
|
Hi @dnhatn, I've created a changelog YAML for you. |
dnhatn
force-pushed
the
match_all_bits
branch
from
d9625d3 to
42ae1b4
Compare
| if (bits == null) { | ||
| initializeBitSet(); | ||
| } | ||
| bits.clear(i); |
There was a problem hiding this comment.
Do we actually need to support clearing bits on this MatchAllBitSet? It's counter intuitive to me that something called MatchAllBitSet could have some of its bits not set.
There was a problem hiding this comment.
Good call. I've pushed 0f12598 to make it read-only.
There was a problem hiding this comment.
This looks fine to me. For the record, I'm in general not a fan of implementing optimizations for cases that should be as rare as this one, but this concern is alleviated by the fact that it shares logic with the same optimization for DLS where I would expect this condition to be less rare.
|
Thanks, Adrien. I was also hesitant to open this PR, but this class already exists in the security module. |
dnhatn
added
the
auto-merge-without-approval
By default, Elasticsearch uses up to 1 bit per document to store the set of root documents when the index mapping has nested fields. This PR introduces a special BitSet using less memory for match_all filters. This optimization is only triggered when the index mapping has a nested field, but that field never exists in documents.
By default, Elasticsearch uses up to 1 bit per document to store the set of root documents when the index mapping has nested fields. This PR introduces a special BitSet using less memory for match_all filters. This optimization is only triggered when the index mapping has a nested field, but that field never exists in documents.